DOCUMENT RESUME 



ED 424 597 



CS 509 925 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Jensen, Gina 

Judges as the Centrality of Forensics: An Examination of 
Norms Sc Training in Forensics Adjudication. 

1997-11-00 

2 5p . ; Paper presented at the Annual Meeting of the National 
Communication Association (83rd, Chicago, IL, November 
19-23, 1997) . 

Reports - Research (143) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

Communication Research; *Debate; Debate Format; Higher 
Education; * Judges; *Oral Interpretation; Pilot Projects 



ABSTRACT 



A pilot study analyzed oral interpretation ballots collected 
at two college forensics tournaments to determine the average number and 
classifications of comments on the ballots. A total of 1,737 comments from 
304 ballots of oral interpretation events (prose, poetry, program, duo, and 
dramatic) were analyzed. Results indicated a mean of 5.71 comments per ballot 
and a total of 25 classifications of comments (including vocal delivery, 
introduction, characterization, emotion, and physical delivery) . Findings 
suggest that interpretation ballots are productive tools for facilitating 
student improvement. On the whole, judges try to use the ballot as an 
educational tool. Further research should focus on specific events in the 
hopes of better understanding each of the oral interpretation arenas. 
(Contains a table of data and 25 references.) (RS) 
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One of the most important forms of feedback a student 
receives in forensic competitions is the ballot. The ballot 
serves to teach, guide and direct the student. It is the litmus 
test for the student's performance. It determines how a student 
finishes at a tournament as well as helping to point the student 
in a direction for future competitions. Pratt (1987) writes in 
his ballot analysis paper that the ballot serves two principal 
functions, "judging and coaching" (1) . He defines judging as 
"evaluating a performance within a competitive context by 
applying a set of external and generally familiar criteria; the 
out come of judging in debate is win-loss decision with speaker 
points and, in individual events is a rank and rating" (1) . He 
goes on to explain that coaching is "evaluating a performance 
within an educational context by making comments on its overall 
quality and by offering suggestions for improvement in later 
performances" (1). Pratt emphasizes that a judge's explanation on 
the ballot may increase the likelihood of understanding. With 
this much importance resting on the ballot, it is imperative that 
a ballot serve its purpose and be written with thoughtfulness and 
care. Unfortunately, some judges take this for granted, others, 
due to a lack of attention, time or experience (or other outside 
interferences) may write ballots that fall short of their 
instructional potential. Poor ballots are those that are not 
helpful to the student in determining why they received the rank 
that they did or how they can further improve their performances. 
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"Nice job" is a positive comment, but it does not really explain 
rank/rate or give suggestions for improvement. 

This paper focuses on a pilot study that analyzes oral 
interpretation ballots collected. It outlines the average 
number and classifications of comments found on ballots. 

Finally, the paper discusses implications of this study on 
judging behaviors in oral interpretation events. 

Literature Review 

A great deal of research focusing on the use of the individual 
event ballots can be found in forensic literature. More 
specifically, a number of studies focus specifically on 
interpretation standards, as well as ballot writing in individual 
events . 

Analysis of Individual Event Ballots: General 

A number of scholars have discussed the comments made by 
judges on ballots and have analyzed the effectiveness of ballot 
content and how to improve a person's ballot writing skills. 

Olson and Wells (1988), stress the importance of a properly 
written ballot. Through their ballot analysis, they determined 
that a good ballot should be thoughtful and thorough and should 
contain a reason for decision. They emphasize that judges need 
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to realize the importance of what they are writing and how the 
ballot will be used, being careful to structure their comments 
towards ideas for improvements. Olson and Wells also cite 
Hanson's (1987) statement of a good judge (based on students' 
opinions) : "A good judge writes comments that are concrete, 
helpful, truthful and are in a sufficient amount that you can 
learn from them" (20) . Olson and Wells then point out that 
following these guidelines will help bolster a judge's 
credibility in the eyes of the competitors. Their method focuses 
on the comments made and their helpfulness and the need for a 
reason for decision on every ballot. They conclude that 
guidelines are necessary in individual events to help better the 
ballot and to help increase the learning that a student can gain 
from reading the ballot. Their final emphasis is for every judge 
to justify all of the comments that they make on the ballot. 

Many other scholars have conducted similar ballot analyses. 
Jensen (1988, 1990) outlines another method in his studies of 
ballot comments. He focuses on the number of comments made on 
original speaking ballots, as well as on the categories of the 
same comments. Each time he choose as his criteria ballots with 
and without event-specific criteria on them. Jensen argues that 
as opportunities continue to open up to students, so too does the 
responsibility and role of the individual events critics. Jensen 
argues that this role is an important one and that judges should 
take responsibility for it. Jensen also agrees that event 




5 



4 



criteria is needed to assist judges in their writing of the 
ballot and their critiquing of the round, but takes the 
suggestion a step further and suggests that guidelines should be 
event specific. Jensen concludes by stating that ultimately it 
is the responsibility of the judge to write a good ballot and 
that most judges are on the "right track to providing students 
with a positive forensics experience" (156). 

Pratt (1987) also examined ballots written for prepared 
speeches in order to "judge the judges." The method that he 
selected enabled him to first count the ballots and code them on 
several levels, and then identify the format in which the ballot 
was written (sentence or sentence fragment) . He coded the topic 
of each unit as one of four mutually exclusive categories. He 
went on to code the ballots for advice and for an explanation of 
ranking and rating. Pratt's effort in analyzing the ballots was 
to discover how the judges are doing when writing their ballots. 
He decided that the judges should receive a grade of B+, as he 
found the ballots on the whole to be "thorough, balanced and 
helpful" (7) . He also suggested that we explain our final 
evaluations more consistently, and to always keep the audience 
members, their needs and their interests in mind as we construct 
our ballots. 

A number of other scholars researched and designed or 
studied methods which are helpful in determine how to write a 
proper ballot and what critics are currently putting on their 
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ballots (Congalton and Olson, 1995; Dean, 1988; Dickmeyer, 1994, 
1997; Hanson, 1987; Hanson, 1989; Jones, 1988; Nicolai, 1987; 
Olson and Wells, 1988; Olson, 1989; Olson, 1992; Perlich, 1994; 
Pratt, 1987; Preston, 1983; Preston, 1990; Renz, 1991; Tucker, 
1988; Wilcox, 1988) . These studies can be useful for judges who 
would like to improve their ballot writing skills. 

Analysis of Individual Events Ballots: Oral Interpretation 

A number of scholars also looked specifically at 
interpretation events and performed ballot analyses to determine 
current practices and methods of improving. Trimble (1994), 
analyzing Cron-Mills' (1991) study, outlines nine guidelines for 
writing an effective ballot. He argues that by following the 
guidelines a person can become a better judge and write a more 
effective ballot for the student. Among the criteria that he 
gives for writing a good ballot are: writing a ballot, "flow" the 
performance, offer comments concerning the competitor's emotional 
and intellectual portrayal of the characters and avoid jargon, 
among others. Trimble explains Cron-Mills' method as one that 
analyzes comments on ballots in an effort to improve judges' 
ballots . 

Nicolai (1989) was interested in coaches' and judges' 
attitudes towards interpretation events and the perceived 
standards that he believes exists. He took a different approach 
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in his methodology; instead of relying on a ballot analysis, he 
used a telephone survey to acquire his results. One of the areas 
that he focused on was judging standards. He wondered if 
differences existed by regions and if there was national 
consensuses regarding events. He also looked at specific 
criteria by event. Nicolai concluded in his study that more 
attention should be given to the distinct character of events. 

Also interested in current practices and methods of 
improving, Dickmeyer (1994) conducted a ballot analysis in order 
to determine that if what should occur on ballots actually does 
occur. Dickmeyer argues that students learn on a number of 
levels and that one of the levels that is many times ignored is 
the ballot. Dickmeyer focuses his paper on what we, as judges, 
should be doing on our ballots. Dickmeyer' s method was to 
interpret each ballot as a unit, as opposed to analyzing each 
comment as a separate unit. He determined, through using Cron- 
Mills/Trimble's guidelines, that no single ballot meets all of 
the criteria. He also states that judges need to address the 
concern that no regard for literature is taken into consideration 
when making performance choices. Dickmeyer further concludes 
that a major concern of coaches and judges is the "lack of 
emotional or intellectual depth an interpreter brings to a 
character, " but that this concern is being ignored on the ballot 
by a large number of judges (11) . Dickmeyer concludes by stating 
that judges should ensure that competitors derive the greatest 
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educational benefits possible from the ballot and to do so, it is 
the responsibility of the judge to take this into consideration 
when constructing their ballot. 

A number of other similar studies have been done to 
determine if judges are executing their duty as a ballot writer 
properly. Scholars have also studied perceived differences and 
practices in interpretation events as well as the changing 
conventions of interpretation. There are a number of places that 
a person can go to extend their review of scholarship in forensic 
interpretation (Aspdal, 1997; Cox, 1989; Knapp, 1997; and Koeppel 
and Morman, 1991) . 



Methodology 

Ballots for this study were collected in the fall of 1997 at 
two forensic tournaments — The University of Missouri-St. Louis 
and Middle Tennessee State University. After collecting ballots 
from all events, the study was focused onto interpretation events 
(prose, poetry, program, duo, and dramatic) . A total of 304 
ballots were analyzed, using a method similar to that employed by 
Preston (1983) . Two pieces of information were recorded during 
analysis: (1) the number of comments made on each ballot, and 

(2) the classification of comments. For the purposes of this 
study, a comment is "any sentence, phrase, or single word that 
provides some critique of the speaker's performance or advice for 
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improvement" (Preston, 1983, p. 2) . As comments were recorded, 
classifications were created, such as 'characterization' or 
' introduction ' in order to determine the types of comments being 
made on interpretation ballots. 

This study seeks to answer two research questions. 

RQ1 What kinds of comments are made on oral interpretation 
ballots? 

RQ2 Do comments made on oral interpretation ballots suggest 
a useful and appropriate criteria for an effective oral 
interpretation performance? 

Results 

A total of 1,737 comments were recorded from the 304 ballots 
analyzed, indicating a mean of 5.71 comments per ballot. 
Classifications of comments were created as ballots were 
analyzed. Each ballot that included a comment within a 
classification was recorded within that comment type. The study 
did not record the overall frequency of comments made per 
classification — only the number of ballots represented within 
each classification. A total of 25 classifications were created. 
While some of these categories appear to be subcategories of 
others, larger categories such as 'physical delivery' and 'vocal 
delivery' were created when critics' comments were 
generalizations. More specific comments resulted in the 
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generation of specific classifications. (See Table One for a 
breakdown of classifications.) 
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Table One 

Classification of Comments on Oral Interpretation Ballots 



Classification of Comment 


Number of 


Percentage of 




Ballots 


Ballots 


Vocal Delivery 


142 


.47 


Introduction 


106 


.39 


Characterization 


106 


.39 


Emotion 


104 


.34 


Physical Delivery 


102 


.34 


Scriptbook 


97 


.32 


Teaser 


86 


.28 


Rate 


79 


.26 


Literature 


53 


.17 


Focal Points 


52 


.17 


Imagery 


51 


.17 


Memorization 


42 


.14 


Ending 


41 


.05 
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Fluency Break 


34 


. 11 


Time 


33 


.11 


Theme 


22 


.07 


Cutting 


21 


.07 


Encouragement 


14 


.05 


Energy 


14 


.05 


Transitions 


14 


.05 


Clothing 


10 


.03 


Message 


8 


.03 


Range 


7 


CM 

O 

• 


Variation in Selections 


6 


CM 

O 

• 


Dialect 


5 


. 02 



When examining RQ1 "What kinds of comments are made on oral 
interpretation ballots", the following information, broken down 
into 25 categories, was found. Judges most frequently made 
comments in the following areas: 

VQCfll Delivery was mentioned on 142 ballots, which is .47 percent 

of the total analyzed ballots. Vocal delivery included any time 
a judge would make mention of vocalization or general vocal 
patterns. Specific vocal patterns received their own category. 

Introductions were mentioned on ballots 106 times which is .39 
percent of the total ballots. Any comment about the introduction 
is included in this category. Sometimes the comments were 
detailed about what a person liked or disliked. Other times 
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comments were one word phrases describing their feelings about 
the introduction. 

Characterization was mentioned on 106 of the ballots, which is 
again .39 percent of the total ballots. Comments were put into 
this category when mention was made about the character in 
general. Comments often described the character's persona as a 
whole, or offered general suggestion for character improvement. 

Emotion was mentioned on 104 of the ballots which is .34 percent 
of the total ballots. Emotion comments included any comments 
dealing with the emotions that the performer is emoting (or not 
emoting) , the lack of emotion (or presence of emotion) that a 
judge is feeling or suggestions as to how a performer can add 
emotion to their piece. 

Physic al Delivery received mention on .34 percent of the ballots, 
which breaks down into 102 comments. Physical delivery comments 
include any comments that were made about physical delivery in 
general. Specific physical delivery comments are represented in 
the other categories. 

Scriptbook received attention .32 percent of the time, or on 97 
ballots. Comments that were placed in this category included 
comments regarding the opening or closing of the book, the way it 
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is being held, that it is being used as a prop, that is isn't 
used often enough, that it should be included in the visual 
aspects of the piece (as a prop) , how page turns are or should be 
executed, that items in the book are distracting, or any other 
mention of the actual book and its pages. 

Teaser was mentioned on 86 ballots which is .28 percent. Like 
the introduction, any mention of the teaser, good or bad, long or 
short, were placed in this category. 

Rate was mentioned .26 percent of the time, or on 79 ballots. 
Although rate could fit within the vocal delivery category, 
specific references to it, resulted in its own category. 

Comments included, "too fast, " "too slow, ” "rate should vary 
throughout piece," etc. 

Literature received attention on 53 ballots, or .17 percent of 
the time. Comments that fit into this category included when a 
judge liked or disliked material, if the literature was too 
challenging, or not challenging enough, or any other general 
reference to the text itself. 

Focal Points were mentioned on 52 ballots, which is .17 percent 

of the total. Comments in this category were again specific to 
focal points, such as making them wider, smaller, or having some. 
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This category also could have fit into physical delivery, but 
since comments were given specific mention, the researcher felt 
that they merited their own category. 

Imagery was found on 51 ballots, or .17 percent of the sample. 

Any comments about imagery in general were included in this 
category. Comments included bringing a piece to life, the need 
for imagery, the need for the student to "see" the piece, and 
many others along those lines. 

Memorization was mentioned on 42 ballots, or .14 percent of the 
time. Any comments about memorization were placed in this 
category. 

Ending of a piece was found on 41 ballots, or .05 percent of the 
total. Comments in this category ranged from "awesome ending" to 
long explanations about how important the right ending is to a 
piece to how an ending could be improved. 

Fluency Breaks were mentioned .11 percent of the time or 34 
ballots. Fluency breaks included any time there was a break or 
stumble in the performance. 

.Time received mention on 33 ballots, which breaks down to be .11 
percent of the time. Comments were placed into this category 
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that included mention about a piece being too short or too long, 
being over time, or any other general mention about time. 

Theme showed up on 22 ballots, or .07 percent of the sample. 
Comments that fit in this category were ones that dealt with the 
themes in poetry or POI . Again comments were both positive and 
negative . 

Cutting comments were found on 21 ballots, or .07 percent of the 
total. Comments relevant to this category were when a judge 
either liked or disliked a particular cutting choice that the 
performer had made or if the judge was familiar with the 
literature and suggested changes. 

Encouragement Comments were found .05 percent of the time, or on 
14 ballots. These comments focused on the performer sticking 
with the event or potential that the judge saw in the person. 
Comments herein are generally uplifting in nature. 

Energy Comments were found on 14 ballots, equalling .05 percent. 
Any comments that mentioned energy specifically were put into 
this category. 

Transitions were mentioned on 14 of the total ballots, or .05 
percent. This category contains comments directed specifically 
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at transitions either between pieces or scene. 

Clothing Comments were made on 10 ballots, or .03 percent. 
Comments in this area included any mention of clothing, earrings, 
nail polish, make-up, professionalism in dress, etc. 

Message was a comment on .03 percent, or 8 of the ballots. This 
category contains statements about messages contained within DI, 
Duo or Prose. 

Range Comments were found on 7 ballots (.02 percent) . These 
remarks dealt with the variety in a student's performance 
choices . 

Variation in Selections was mentioned on 6 ballots for a total of 
.02 percent. Any suggestion that pieces were heard frequently, 
or that selections in a program were too similar were put into 
this category. 

Dialect Comments were on .02 percent of the ballots (5 total 
ballots) . These comments included any remarks about accents. 

When examining RQ2, "Do comments made on oral interpretation 
ballots suggest a useful and appropriate criteria for an 
effective oral interpretation performance, " the researcher 
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determined, based on remarks found on ballots, that most of the 
time comments do suggest a useful and appropriate criteria for an 
effective interpretation performance. At times, comments were 
found to be too vague for a performer to use in any way. At 
other times, the critics' lack of writing a ballot served as a 
hinderance. Some ballots only contained the dreaded "good job, 
5-15" remarks. Upon overall examination of the ballots, the 
majority of adjudicators made at least some effort to help 
students to improve upon their performance. There were times 
when only certain aspects of what others have listed as criteria 
for a good ballot were met. Sometimes critics would only focus 
on one part of the performance, leaving the other facets 
unmentioned. As a whole, though, it appeared as if judges made 
an effort to suggest helpful criteria. 



Discussion 

When looking back at the different studies that have been 
done, this one is in agreement with existing literature. The 
researcher concluded that on a whole judges are doing a pretty 
good job, but that there is room for improvement. After 
surveying the ballots for comments, the researcher concurs with 
Trimble's analysis that not many judges meet all of the nine 
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guidelines that Cron-Mills outlines. In fact, some only comply 
with one or two of the guidelines. For a number of reasons, 
judges often times do not write a "complete" ballot. A judge may 
be distracted with outside forces such as hunger or fatigue which 
may cause them to write an underdeveloped ballot. Pratt argues 
that many judges write their ballots immediately after each 
performance and then wait until the end of the round to rank and 
rate each person. He explains that it is unlikely that a judge 
will go back and further explain their ballot or write a reason 
for decision, usually because of tight tournament schedules (6). 
Even though some ballots have rooms for improvement, all in all 
interpretation ballots prove to be productive tools for 
facilitating student improvement. On the whole, judges try to 
use the ballot as an educational tool and even though some leave 
a bit to be desired, others make up for it. Plus, even if some 
ballots do not meet all of the guidelines, at least as a starting 
point, some productive comments are better than a blank ballot. 

It is uplifting that critics are striving to write good 
ballots, but judges must remember that it is an on going process. 
A person should always strive to improve their ballot skills. 

For instance, it is gratifying that only a small percentage of 
comments are directed at clothing in the study. However, remarks 
that are generalized such as vocal delivery are problematic. 

Vocal delivery supported by examples and explained explicitly are 
appropriate comments to make, but unexplained and not supported 
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with examples, make that comment vague and hard to adapt to. 
Critic should strive to be as specific as possible with regards 
to their criticism. Results of this study suggest that such 
specificity is often lacking in interpretation ballots. 

An informal finding not outlined in this study is that 
judges seem to repeat the same comments for different competitors 
numerous times. Not only in the same section, but on most of the 
ballots that they write. Leading a person to wonder how much 
thought goes into each individual ballot and each individual 
performance. Some people seem to fall into comfortable patterns 
and not want to stray from that norm. 

Further research should focus on specific events in the 
hopes of better understanding each of our oral interpretation 
arenas . 

As mentioned earlier, Hanson outlines the most important 
traits associated with a good judge, "writes concrete, helpful, 
truthful comments in a sufficient amount that you can learn from 
them" (20) . He also states that it is "important for judges to 
try even harder to offer some comments which can lead to growth 
opportunities for the contestant" (18) . In order to write a good 
ballot, a judge should take these ideas into consideration. A 
critic should also support their remarks with explanation and 
examples, they should take the necessary time needed to write a 
thoughtful and helpful ballot, not worrying about being the first 
to finish. They should also write a reason for decision so that 
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a student not only knows how their performance was, but why they 
received the rank and rate that they did. Finally, in 
constructing a good ballot, an adjudicator should take into 
consideration Cron-Mills nine guidelines (1. write a ballot, 2. 
divulge your philosophy of interpretation, 3. suspend evaluations 
of "literary merit" or "past experience", 4. "flow" the 
performance, 5. offer comments concerning the technical aspects 
of the presentation, 6. offer comments concerning the 
competitor's emotional/ intellectual portrayal of the characters, 
7. avoid jargon, 8. don't ignore the primary issues, and 9. 
include constructive criticism) . If a judge keeps all (or even 
part) of these suggestions in mind, as well as the education and 
growth of the students, they should be able to write an effective 
ballot . 

Summary 

When analyzing ballots collected from two tournaments and 
considering the other studies that have been done, it is apparent 
how important the construction of the ballot is. The ballot is 
the only thing that a competitor has to evaluate their 
performance at tournaments and if the judge does not take the 
needed time to write a thorough ballot, the student's performance 
suffers, a coaches ability to coach is decreased and in general, 
pedagogy suffers. While it may seem like an unimportant task to 
some when they are in a hurry, tired or hungry, the lack of a 
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properly written ballot can definitely hurt the student. Judges 
should take into consideration the need to write an informative 
ballot, one that follows criteria for the event, lists a reason 
for decision and that gives explanation for comments and judges' 
preference. So next time you are in a 8 am round and you skipped 
breakfast, remember that not writing an effective ballot is like 
your not having been there at all, so you would have gotten up at 
the crack of dawn and starved yourself for nothing. 
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